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Abstract- In this paper, we have discussed the basics of 
augmented reality, its functionality, its working and its 
application in the field of education for children with learning 
disabilities or special needs. Augmented reality provides a very 
interactive, vibrant and insightful aspect to traditional learning. 
It allows combining a myriad multimedia objects; such as, 
images, videos, audio files and three dimensional object models. 
To further understand its objective, we have also explained the 
multiple learning disabilities and how this education tool serves 
as a better education tool. Depending on the type of learning 
disability, different kids have different educational requirements. 
Some disabilities cause them to have a range of emotional 
responses based on application components like colors and the 
interaction interface, etc. Therefore, special consideration has to 
be given to the design and interface of educational tools targeted 
at these children. Special focus is laid on its performance ability 
and current status of technology oriented education tool in India. 
We have verified this by creating a prototype education tool as 
per the needs of two educational organizations in Mumbai and 
testing the tool in an experimental class to analyze the impact and 
limitations. Our education tool comprehensively addresses the 
diverse range of factors that need to be kept in mind while 
designing educational applications for children with the specific 
learning disabilities. In this paper, we explain the specific 
components of our application using which we have achieved the 
totality required for an education tool that deals with an 
especially sensitive class of students. 

Index Terms- Augmented reality, android mobile application, 
children, education in India, image processing, learning 
disabilities, mobile education, multimedia teaching tools, object 
recognition 

I. INTRODUCTION TO AUGMENTED REALITY 

Augmented reality (AR) is live immediate or roundabout 
perspectives of a physical, genuine environment whose 
components are increased (or supplemented) by PC produced 
tangible enter, for example, sound, feature, representation or 
GPS information. It is identified with a more general idea 
called intervened reality, in which a perspective of the truth is 
adjusted (perhaps even decreased as opposed to expanded), by 
a PC. Therefore, the innovation capacities by upgrading one's 
present view of reality. 
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By differentiation, virtual reality replaces this present reality 
with a reenacted one. Enlargement is ordinarily continuously 
and in semantic connection with natural components, for 
example, games scores on TV amid a match. With the 
assistance of cutting edge augmented reality innovation (e.g. 
including PC vision and item distinguishing) the data about 
the encompassing true of the client gets to be intuitive and 
digitally manipulative. Simulated data about nature and its 
protests can be overlaid on this present reality. 

II. INTRODUCTION TO LEARNING DISABILITY 
AND CHILD DEVELOPMENT 

An issue in one or a greater amount of the fundamental mental 
procedures included in understanding or in utilizing dialect, 
talked or composed, that may show itself in the blemished 
capacity to tune in, think, talk, read, compose, spell, or to do 
numerical estimations. 

In India 1.67% of the 0-19 population has a disability. 35.29% 
of all people living with disabilities are children. Other 
estimates say that India has 12 million children living with 
disabilities. Only 1% of children with disabilities have access 
to school [3]. 

Training for the individuals with incapacities/uncommon 
needs has not been profoundly offered thought in developing 
nations like India and is likewise joined by monetary issues 
and asset supports to advance instruction and giving a 
possibility of work to them. In spite of the fact that US has 
discovered innovation propelled instruction help to help add to 
an educational module, Indian schools do not have this 
preference and advancement. 

Turning into a capable independent citizen is crucial for 
conforming to grown-up life, yet numerous grown-ups with 
learning inabilities (LD) rank taking care of cash and keeping 
money as the most troublesome among the issues they 
experience. Issues here are regularly fixed to particular 
attributes of LD or attention deficit/hyperactivity disorder 
(AD/HD) and other extraordinary needs like Speech or 
Language Impairment (SLI) correspondence issue, for 
example, faltering, impeded explanation, a dialect impedance, 
or a voice debilitation that unfavorably influences a tyke's 
instructive execution and Specific Learning Disability (SLD). 

The accompanying are so a percentage of the numerous 
ailment and incapacities confronted by youngsters that can be 
bolstered with legitimate training frameworks. 
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The following are so some of the many illness and disabilities 
faced by children that can be supported with proper education 
systems, as studied at our associated schools mentioned later 
in the paper [4] - 

A. Autism (AU) - A formative incapacity altogether 
influencing verbal and nonverbal correspondence and 
social connection, for the most part present before age 
three that unfavorably influences a kid's instructive 
execution. Different qualities regularly connected with 
extreme introvert characteristics are taking part in tedious 
exercises and stereotyped developments, imperviousness 
to ecological change or change in day by day schedules, 
and abnormal reactions to tangible encounters. 

B. Developmental Delay (DD) - As characterized by every 
State, implies a deferral in one or a greater amount of the 
accompanying ranges: Physical advancement, cognitive 
(discernment) improvement, correspondence, social or 
enthusiastic advancement, or versatile (day by day living 
abilities) improvement. 

C. Emotional Disturbance (ED) - A condition showing one 
or a greater amount of the accompanying qualities more 
than a drawn out stretch of time and to a checked degree 
that unfavorably influences a kid's instructive execution: 

A powerlessness to discover that can't be clarified by 
educated, tangible, or wellbeing components. 

A powerlessness to manufacture or keep up agreeable 
interpersonal associations with associates (different 
understudies) and educators. 

D. Multiple Disabilities (MD) - Simultaneous (happening in 
the meantime) weaknesses, (for example, scholarly 
inability lack of sight, intelligent handicap orthopedic 
hindrance, and so forth.), the mix of which causes such 
extreme instructive needs that they can't be served in a 
custom curriculum program exclusively for one of the 
debilitations. The term does exclude hard of hearing 
sightlessness. 

E. Specific Learning Disability (SLD) - An issue in one or a 
greater amount of the essential mental courses of action 
included in understanding or in utilizing dialect, talked or 
composed, that may show itself in the flawed capacity to 
tune in, think, talk, read, compose, spell, or to do 
numerical computations. The term incorporates such 
conditions as perceptual inabilities, mind damage, 
insignificant cerebrum brokenness, dyslexia, and 
formative aphasia. The term does exclude learning issues 
that are principally the aftereffect of visual, hearing, or 
engine incapacities; of learned handicap; of passionate 
unsettling influence; or of ecological, social, or monetary 
burden. 

F. Speech or Language Impairment (SLI) - A 
correspondence issue, for example, faltering, disabled 
explanation, dialect impedance, or voice weakness that 
unfavorably influences a tyke's instructive execution 

Research demonstrates that low instructive 
accomplishment, job desires and mistaking legislative 
projects for clashing qualification criteria have brought 
about numerous youngsters with incapacities not 


making effective moves from school to postsecondary 
instruction, vocation and independent living. While 
numerous might want to figure out how to spare cash 
and manufacture resources, they apprehension landing 
a position and sparing a share of their salary may make 
them lose their incapacity advantages and different 
backings, for example, medicinal services. One 
noteworthy hindrance that adds to this issue is the 
absence of useful math (ideas of time, cash, weights 
and measures, and so on.) abilities among this 
gathering. To add to this officially existing issue, there 
is the issue of English being the internationally 
acknowledged medium of composed and verbal 
correspondence in all available study material. 

III. APPLICATION OF AUGMENTED REALITY IN 

EDUCATION 

Augmented reality applications can complement a standard 
curriculum. Text, graphics, video and audio can be 
superimposed into a student’s real time environment. 
Textbooks, flashcards and other educational reading material 
can contain embedded “markers” that, when scanned by an 
augmented reality device, produce supplementary information 
to the student rendered in a multimedia format. Students can 
participate interactively with computer generated simulations 
of historical events, exploring and learning details of each 
significant area of the event site. On higher education, there 
are some applications that can be used. For instance, 
Construct3 D, a Studiers tube system, allows students to learn 
mechanical engineering concepts, math or geometry. This is 
an active learning process in which students learn to learn with 
technology. Augmented reality can aid students in 
understanding chemistry by allowing them to visualize the 
spatial structure of a molecule and interact with a virtual 
model of it that appears, in a camera image, positioned at a 
marker held in their hand. It can also enable students of 
physiology to visualize different systems of the human body in 
three dimensions. Augmented reality technology also permits 
learning via remote collaboration, in which students and 
instructors not at the same physical location can share a 
common virtual learning environment populated by virtual 
objects and learning materials and interact with another within 
that setting. 

This resource could also take of advantage in Primary School. 
Students learn through experiences, besides when children are 
so young, they need see to learn. For instance, they can learn 
new knowledge about Astronomy, which is usually difficult to 
acquire to them, with this device children can understand 
better The Solar System because they would see it in 3D; even 
children under 6 years old could understand it following that 
method. In addition, learners could change the pictures of their 
Science Book for using this resource. On the other hand to 
teach bones or organs, they could also stick one paper on their 
body and that paper contains an embedded “markers” about a 
bones or an organ that existed under the paper, and the teacher 
would only need to press a button when children would 
change the place of the paper, in this way, we would use the 
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same embedded “markers” in order to teach another part of the 
body. 

Develop the value of the practical module implementation in 
schools and organizations. To provide the support required to 
develop a need based curriculum using technology to close the 
gap between societal problems for people with disabilities. 
Since learning seems to be the main cause of concern, 
technology can be used to develop innovative teaching 
programs that help in faster rate of learning, better memory 
and grasping power. 

Augmented Reality proves its results for children with 
learning disabilities as it directly works on the different 
difficulties they face, from being shy to talk to humans, lack of 
attention span, need for multimedia visuals and environment 
for better grasping ability and easy memorizing ability. We 
have conducted and tested the results for the same further in 
this paper. 


IV. CURRENT STATE OF AUGMENTED REALITY 
APPLICATION IN EDUCATION IN INDIA 

Trimensions is a software development firm providing 
services on augmented reality [5]. Trimensions is into 
developing educational and learning systems using 
Augmented Reality, on the 3D Web as well as on popular 
virtual worlds like Second Life and platforms like OpenSim 
and Unity. However, their official website does not list any 
augmented reality application to provide education to either 
normal kids in pre-school and primary school or kids with 
learning disabilities. Same is the case with other software 
firms that have just begun their venture into augmented 
reality. Some Indian augmented reality applications which are 
built with a far-sighted educational perspective are not based 
on the Android platform. They are for desktops or laptops, etc. 
The rare educational applications using augmented reality 
which are based on the Android platform are expensive and do 
not have a comprehensive focus including considerations for 
kids with special needs; e.g., those suffering from learning 
disabilities. 

In India, the fresh concept of augmented reality has just begun 
penetrating not only various old and new software companies 
but also educational institutions providing technical courses on 
computer technologies. Its main attraction for the developers 
is games, entertainment, language, geophysical mapping, etc. 
It will take time for it to reach the level of circularity it needs 
to address comprehensively the issues involved in providing 
education to all categories of kids. In our application, we have 
tried to inculcate this same comprehensiveness we speak of. 


V. CURRENT TEACHING METHODS IN INDIA 
FOR CHILDREN WITH LEARNING DISABIILITIES 


Augmented Reality has been used by a technology services 
enterprise in the US (Alive Studio Co) to create storybooks for 
kindergarten children, teaching basic number skills to 
preschoolers etc [1]. But so far, there is no recognized 
augmented reality tool used for education for children with 
learning disabilities. Certain technology tools such as 
EduComp Smart class has been used in certain schools it is a 
teacher-led educational content based solution that has 
dramatically improved learning outcomes in Private Schools 
[2]. It comprises of projector screens with digital curriculum 
content. Most schools for special children still stick to paper 
aid for teaching. 

Currently Education is provided at a very basic schooling level 
in certain schools for people with disabilities. Finance 
education is ignored by most schools in India as an 
unnecessary subject, but its need grows with the development 
and empowerment of the country and its people. Augmented 
Reality technology to develop an android application on 
money management (Financial Literacy) specially designed 
for people with special needs (Learning Disabilities) would be 
a significant solution to an emerging fundamental learning 
disability subject. Augmented Reality helps address various 
learning disabilities helping the students learn. The concept of 
using augmented reality - 3D Modeling- Audio Video and 
Images, helps generate a greater engaging time span with the 
students using a highly cost-effective project. 

VI. WORKING OF ARET 

Our project comprises of a simple Smartphone holder, a table 
to rest the books/flashcards and board games, a cable 
connecting the phone to the projected screen and speakers for 
audio output. Through the project, the teacher is able to read 
3D books, play interactive games and flashcard pop-up quiz 
with the class. The ability to impact the sense of sight and 
hearing triggers a vital enhancement in the education level for 
the children as proved by research. This mobile kit can be 
used anywhere without large resource or material 
requirements. Our application has 3 major modules - 
alphabets, numbers and shapes each witth audio, video and 3D 
models. 

We created a prototype, called Augmented Reality Education 
Tool (ARET) to test the concept of augmented reality and 
analyze its impact, outcome and limitations. It makes the use 
of Vuforia SDK developed by Qualcomm Connected 
Experiences, Inc. [7] Using the SDK along with Unity 3D [8], 
we created Augmented Reality applications. 

Using Vuforia SDK, we created flash cards with tracking 
points. These tracking points help the application to identify 
the correct alphabet or the number and augment the 
corresponding objects. These tracking points are unique to 
each flash card. The more the tracking points, the better the 
application can track it. These tracking points are calculated 
based on the difference between the neighboring pixels. If the 
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selected pixel has a higher intensity than a predefined 
threshold after comparing it to the neighboring pixels, that 



Fig. 1 Alphabet Flashcards used in ARET 

pixel is selected as a tracking point. The points are also 
calculated along the edges of any point by again calculating 
the pixel intensities along the edge. 


AURGEMTED REALITY EDUCATION TOOL 



AWV Flash cards 

A- Alphabet image with 
Audio 




W- Word 3D model 
With Audio 



V- Sentence Video 
with Audio 


TOOL NOliVOnOB A1ITV3U Q31W39dnV 


Fig. 2 (a) Virtual Buttons in alphabet flashcard 




A - Number image W - Word 3D model V - Sentence video 
with audio with Audio with audio 


After calculating unique points for each flash card, the files 
are imported to Unity 3D for further processing. Along with 
the files from Vuforia, we also import 3D models that were 
modeled in different modelling softwares, the audio files and 
the video files that are needed in the application. We provide 
the behaviors of each element like the virtual buttons, audio, 
video and the 3D models by writing code for each event. After 
adding the behaviors for each event, we build the application 
using the Android SDK and get the apk file. This apk file is 
used to run the application on the mobile device. 

The ARET application consists of 3 modules: 

• Alphabets 

• Numbers 

• Shapes 

Each of which uses different functionalities of augmented 
reality. The multimedia data in each comprises of: 

• Text 

• Audio 

• Video 

• Image 

• 3D Models 


Fig. 2 (b) Virtual Buttons in number flashcard 

All modules are interactive, as the AWV buttons can be 
touched to display the data stores, the card can be rotated to 
view different angles of the object; in the case of shape cards, 
the output answer text is shown on the card space below. 



AUGMENTED REALITY EDUCATION TOOL 


FOOTBALL 



Fig. 3 (a) Demo run of Shape Circle with text displayed in box 
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Fig. 3 (b) Demo run of number 1 



Fig. 3 (c) Demo run of alphabet C 


VII. SURVEY AND ANALYSIS 

We have associated with CHILDReach school, Mother 
Foundation and community centres of Aastha Parivaar for our 
project [6]. 

CHILDReach is an NGO school for students with learning 
difficulties that range from dyslexia, behavioral problems and 


ADHD. CHILDReach set up a program for children with 
learning difficulties in Santacruz (West), Mumbai in 1999. It 
advocates appropriate education for the students with learning 
difficulties and slow learners. CHILDReach caters to children 
in the age group of 6 to 18 years. It has an ideal student- 
teacher ratio of 8:1 that ensures a lot of personal support. 
Currently, over 60 students from a cross section of society 
study at CHILDReach. 

Mother foundation is a boys’ orphanage currently Located in 
Malad (West). It houses approximately 50 boys of different 
age groups ranging from 3 to 16 years of age and the boys 
attend a local Hindi medium school. 

Below are the details of the survey we conducted in our 
associated centres - 

Current Education level: Kindergarten 
Current education Language: Hindi and English (secondary) 
Medium of education (tools/teaching aid used): Speaking, 
writing, boards, stories 

Current English language level: Extremely basic (few words 
and numbers only) 

Words taught using ARET: 10 words - Apple, Cat, Dog, 
Goose, Hat, Kite, Lion, Mouse, Orange and Queen 

Total time of teaching words with and without ARET each: 30 
minutes and 20 minutes respectively 

Quantitative results: 


Table. I Education Efficiency 


% of words learnt 

Education Efficiency 

Age <= 3 

3 < Age < 6 

Verbal Teaching 

60 

60 

Using ARET 

80 

90 


Table. II Level of Development 


% of Children 
who learnt the 
words 

Level of Development 

Age <= 3 

3 < Age < 6 

Verbal 

Teaching 

63.64 

54.55 

Using ARET 

100 

90.91 
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Qualitative results: 


Survey Conclusion: 


Table. Ill Level of Interest 


Rating (%) 

Level of Interest 

Age <= 3 

3 < Age < 6 

Verbal Teaching 

90 

60 

Using ARET 

100 

80 


Age group with most interest in using ARET: 
Age<=3 

Age group that learnt most no. of words: 3<Age<6 
Age group that most benefited from it: Age<=3 
Overall improvement in pronunciation (Rating 
UA/VT): 33% 

Overall improvement in learning efficiency (Rating 
UA/VT): 41.7% 

Overall improvement in interest in education (Rating 
UA/VT): 16.7% 


Table. IV Pronunciation 


VIII. LIMITATIONS AND FUTURE SCOPE 


Rating (%) 

Pronunciation/ Diction 

Age < 3 

3 < Age < 6 

Verbal Teaching 

60 

60 

Using ARET 

80 

80 


Age <= 3 

120 
100 
80 
60 
40 
20 
0 

Education Level of Level of Pronunciation 
Efficiency Interest Development 

■ Verbal Teaching BUsing ARET 



Although, we have come a long way through our endeavor at 
providing a comprehensive approach to educating kids using 
augmented reality in India, the idea is novel. Hence, there is a 
long road ahead of us to try and polish our application into one 
that is at a universally accepted quality standard. ARET, for 
now, is a basic toolkit limited to letters, numbers and shapes. 
There are myriad components, both software and hardware, 
that can be added to our application in the future. We can 
extend the letters to a proper language module, the numbers to 
a mathematics module, and so on. 

Project ARET, in the future, also aims to overcome the 
limitation of being inflexible. We hope to make the 
application increasingly customizable and interactive. We can 
add games and quizzes, 3D videos, stories, evaluation tools, 
augmented reality spaces within the application, etc. 

A key factor for the creation of Project ARET was 
affordability; therefore, arising compromises because of that 
also need to be addressed. We have to aim to provide a high 
quality application at a low price so that every person in every 
nook and cranny can use it. In comparison to the augmented 
reality product for children in USA, ARET costs l/60 th the 
price. 


Fig. 4 Performance of children aged less than 3 


3 < Age < 6 




■ Verbal Teaching 5 Using ARET 


Fig. 5 Performance of children aged above 3 


We also have a lot of concerns regarding optimization of the 
application’s performance parameters, i.e. processor usage, 
memory space required, response time, camera sensitivity, 
effect of light, and heat generation. 

In the future, we hope to make ARET one of the best choices 
for teaching kids using augmented reality, especially those 
with learning disabilities. For this purpose, we will collect 
more sample data and conduct a pilot study on other social, 
mental and behavioral patterns we need to incorporate in our 
algorithms so ARET is closer to classroom standards for all 
categories of kids. 
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